1. Technical Field
The present invention relates to driver assistance systems in motor vehicles, and in particular to providing a system and a method for night vision using a near infra-red (NIR) Illumination and a rolling shutter for driver assistance systems (DAS) applications.
2. Description of Related Art
During the last few years camera based driver assistance systems (DAS) have been entering the market; including lane departure warning (LDW), automatic high-beam control (AHC), traffic sign recognition (TSR) and forward collision warning (FCW).
Lane departure warning (LDW) systems may be designed to give a warning in the case of unintentional lane departure. The warning is given when the vehicle crosses or is about to cross the lane marker. Driver intention is determined based on use of turn signals, change in steering wheel angle, vehicle speed and brake activation. There may be various LDW systems available. One algorithm for lane departure warning (LDW) used by the Applicant/assignee (Mobileye Technologies Ltd., Nicosia, Cyprus, hereinafter “Mobileye”) of the present application is predictive in that it computes time-to-lane crossing (TLC) based on change in wheel-to-lane distance and warns when the time-to-lane crossing (TLC) is below a certain threshold. Other algorithms give a warning if the wheel is inside a certain zone around the lane marker. In either case, essential to the lane departure warning system is the lane marker detection algorithm. Typically, the lane markers may be detected in the camera image and then, given the known camera geometry and camera location relative to the vehicle, the position of the vehicle relative to the lane is computed. The lane markers detected in the camera image may be then collected over time, for instance using a Kalman filter. Wheel-to-lane marker distance may be given with an accuracy of better than 5 centimeters. With a forward looking camera, wheel-to-lane marker distance is not observed directly but is extrapolated from the forward view of the camera. The closer road markings may be observed, less extrapolation is required for determining wheel-to-lane marker distance and more accurate estimates of wheel-to-lane marker distance may be achieved especially on curves of the road. Due to the car hood and the location of the camera, the road is seldom visible closer than six meters in front of the wheels of the car. In some cars with longer hoods, minimal distance to visible road in front of the car is even greater. Typically the lane departure warning system of Mobileye works on sharp curves (with radius down to 125 m). With a horizontal field of view (FOV) of 39 degrees of the camera, the inner lane markers may be still visible on curves with a radius down to 125 meters. In order to correctly perform lane assignment on curves, lane markings may be detected at 50 meters and beyond. With a horizontal field of view (FOV) of 39 degrees for the camera, a lane mark of width 0.1 meters at 50 m distance corresponds in the image plane to just under two pixels wide and can be detected accurately. The expectation from the lane departure warning systems is greater than 99% availability when lane markings may be visible. Expectation with 99% availability is particularly challenging to achieve in low light conditions when the lane markings may be not freshly painted (have low contrast with the road) and the only light source is the car halogen headlights. In low light conditions, the lane markings may be only visible using the higher sensitivity of the clear pixels (i.e. using a monochrome sensor or a red/clear sensor). With the more powerful xenon high intensity discharge (HID) headlights it is possible to use a standard red green blue (RGB) sensor in most low light conditions.
Traffic sign recognition (TSR) modules may be designed typically to detect speed limit signs and end-of-speed limit signs on highways, country roads and urban settings. Partially occluded, slightly twisted and rotated traffic signs may be preferably detected. Systems implementing traffic sign recognition (TSR) may or should ignore the following signs: signs on truck/buses, exit road numbers, minimum speed signs, and embedded signs. A traffic sign recognition (TSR) module which focuses on speed limit signs does not have a specific detection range requirement because speed limit signs only need to be detected before they leave the image. An example of a difficult traffic sign to detect is a 0.8 meter diameter traffic sign on the side of the road when the vehicle is driving in the center lane of a three lane highway. Further details of a TSR system is disclosed by the present assignee in patent application publication US20080137908.
A typical automatic headlight or high/low beam control (AHC) system detects the following conditions and switches from high beams to low beams: headlights of oncoming vehicles, taillights of preceding vehicles, street lights or ambient light indicating that high beams may be not required and a low vehicle speed. The host vehicle lights may be switched back to high beams when none of these conditions exist (often after a specified grace period). One approach for detecting taillights is to compare images from two sensors: one with a red filter and the second with a cyan filter. The cyan filter responds to non-red light sources and will give zero response to red light. By comparing corresponding pixels from two imaging sensors one can detect the color of the light source. The number of pixels of each color above a certain intensity is counted and if the count is above a threshold the systems switches to low beams. The use of color filters with imaging sensors may preclude the simultaneous use of the same image frames for other driver assistance applications.
A second approach for automatic high-beam control (AHC) uses an RGB sensor to give better color differentiation. Typical light sources can be located in the full CIE color space as defined by the International Commission on Illumination. This approach distinguishes between green, yellow and red lights. A powerful green traffic light is not confused with an oncoming vehicle. Since a single sensor with a color mosaic filter i.e. Bayer pattern mosaic is used, the lens is defocused so as to spread a light source over multiple pixels. The use of the color mosaic filter reduces both the effective image sensor resolution (by 50%) and the intensity response (to less than one third). The color mosaic filter may preclude the use of the same sensor for traffic sign recognition (TSR) or lane departure warning (LDW) because of the intensity response penalty.
Ego-motion estimation is disclosed in U.S. Pat. No. 6,704,621 by Stein. Image information is received from images recorded as the vehicle moves along a roadway. The image information is processed to generate an ego-motion estimate of the vehicle, including the translation of the vehicle in the forward direction and the rotation.
Some driver assistance systems may rely on changing camera exposure parameters (e.g. aperture, exposure, magnification). The use of a color camera equipped for instance with an RGB (red/green/blue) filter and an infra-red filter achieves good spectral separation for detecting taillights or brake lights but reduces imaging sensitivity by a factor of six or more. A reduction in sensitivity by such a factor has an adverse impact on other vehicle control application such as LDW performance in dark scenes. The presence of an infra-red filter also negates the use of the camera as a near infra-red sensor for applications, such as pedestrian detection. Thus, a brake light detection system which uses color or spectral analysis in the captured images may be less compatible with other driver assistance systems without sacrificing performance.
The core technology behind forward collision warning (FCW) systems and headway distance monitoring is vehicle detection. Assume that reliable detection of vehicles in a single image a typical forward collision warning (FCW) system requires that a vehicle image be 13 pixels wide, then for a car of width 1.6 m, a typical camera (640×480 resolution and 40 deg FOV) gives initial detection at 115 m and multi-frame approval at 100 m. A narrower horizontal field of view (FOV) for the camera gives a greater detection range however; the narrower horizontal field of view (FOV) will reduce the ability to detect passing and cutting-in vehicles. A horizontal field of view (FOV) of around 40 degrees was found by Mobileye to be almost optimal (in road tests conducted with a camera) given the image sensor resolution and dimensions. A key component of a typical forward collision warning (FCW) algorithm is the estimation of distance from a single camera and the estimation of scale change from the time-to-contact/collision (TTC) as disclosed for example in U.S. Pat. No. 7,113,867.